The invention relates generally to an improved method for locating and reading two-dimensional barcodes printed within an image.
Contrary to the frequent predictions that we will one day live in a xe2x80x9cpaperless societyxe2x80x9d, paper, and other printed mediums, are playing an increasingly important role as an inexpensive, effective and convenient means for communication. A fundamental limitation with paper, however, is that from a computer standpoint, it is currently an output-only format. While paper may be the preferred medium for displaying information for human use, it is difficult, if not impossible, for a computer to recover data reliably once it has been printed. Optical character recognition (OCR) attempts to solve this problem in a relatively simple domain, such as text rendered using standard fonts, but has met with only limited success thus far. While accuracy rates of ninety-nine (99%) percent are perhaps achievable and may seem impressive, a page with 3,000 characters will still incur an average of thirty (30) OCR errors and hence requires expensive and time consuming manual post-processing.
Another approach uses computer readable barcodes which may be included directly on paper (or other printed medium such as microfilm). Once encoded, such barcodes can be used by the computer to recover information evident to the human reader but difficult for a computer to recognize (e.g., printed text), information implicit to the creation of page but essentially invisible to the human reader (e.g., spreadsheet formulas), or any other information desired, whether or not dependent on the actual character text on the paper.
Computer readable barcodes, wherein digital data is recorded directly on paper, are known and have been utilized to provide document or product identification given a fixed set of values using simple numeric encoding and scanning technologies. Document or product identification systems which have been employed in the past include barcode markers and scanners which have found use in a wide range of arenas. With respect to paper documents, special marks or patterns in the paper have been used to provide information to a related piece of equipment, for example the job control sheet for image processing as taught by Hikawa in U.S. Pat. No. 5,051,779. Similarly, identifying marks comprising encoded information have been printed on the face of preprinted forms as described in U.S. Pat. No. 5,060,980 to Johnson, et al. The Johnson, et al. system provides for a user entering hand drawn information in the fields on a paper copy of the form and then scanning the form to provide insertions to the fields in the duplicate form that is stored electronically in the computer. Still another system is described in U.S. Pat. No. 5,091,966 of Bloomberg, et al., which teaches the decoding of glyph shape codes, which codes are digitally encoded data on paper. The identifying codes can be read by a computer and thereby facilitate computer handling of the document, such as identifying, retrieving and transmitting such document.
Besides the various shaped barcodes described above, two-dimensional barcodes called xe2x80x9cdata stripsxe2x80x9d having a plurality of rows of xe2x80x9cdata linesxe2x80x9d that represent information digitally encoded on printed media are also known in the art. Each data line row consists of a series of black and white pixels each representing binary xe2x80x9c0xe2x80x9ds and xe2x80x9c1xe2x80x9ds. The ordering of the bits in each row determines the digital data stored therein. The data stored within the totality of the rows define the data contained in the two-dimensional barcode. Typically, to read the barcode, the user passes a hand scanner, which simultaneously reads the information in each data line row, vertically along the length of the barcode to read all of the data line rows.
An example of a prior art system using a data strip two-dimensional barcode having rows of data lines with paper media, is found in U.S. Pat. Nos. 4,692,603, 4,754,127 and 4,782,221 of Brass, et al. In this system, two-dimensional barcodes consist of data line rows which are used to encode computer programs and data on paper and are scanned by use of a hand scanner. In addition to encoding the computer programs and data, these data lines also contain tracking and synchronization bits, hereinafter referred to as xe2x80x9cclock bitsxe2x80x9d. The requirement for use of numerous clock bits directly within each data line row, significantly reduces the amount of digital data that can be stored within each row. Further, if data line rows having clock bits are damaged, which is common if such barcodes are photocopied or transmitted by facsimile systems, such clock bits would be lost making it difficult, if not impossible, to decode the information encoded in the barcode. Other examples of two-dimensional barcodes include: (1) U.S. Pat. No. 5,083,214 to Knowles, which describes a two-dimensional barcode system that requires clock bits embedded within the encoded data itself; and (2) U.S. Pat. No. 4,924,078 to Sant""Anselmo et al., which describes a two-dimensional barcode system in which an orientation and/or timing cell border is included within the body of the barcode itself.
In addition, in co-pending patent application xe2x80x9cA Clock-Free Two-Dimensional Barcode and Method for Printing and Reading the Samexe2x80x9d, (Ser. No. 08/569,280, filed Dec. 8, 1995) (xe2x80x9cthe ""280 Applicationxe2x80x9d), the contents of which are explicitly incorporated by reference herein, a clock-less two-dimensional barcode with a border on at least one of the four sides of the barcode is described, which border is placed outside the confines of the barcode itself. The two-dimensional barcodes are sometimes called xe2x80x9cPanaMarksxe2x80x9d(copyright). As depicted in FIG. 1A herein, two-dimensional barcode 10 is printed in the low right hand comer of printed page 11, although this position is completely arbitrary. In the embodiment depicted in FIG. 1A, the remaining portion of printed page 11 is occupied by printed text 12. However, as one skilled in the art will appreciate, any type of computer-generated printed material, for example a spreadsheet or graphics, can be substituted for the printed text 12. The two-dimensional barcode 10 depicted in FIG. 1B herein includes a border 13 that is present on all four of its sides. As is fully described in the ""280 Application, although the border 13 is only needed on one of the four sides of the two-dimensional barcode 10, for aesthetic reasons it is typically included on all four sides.
Also, in co-pending patent application xe2x80x9cA Borderless Clock-Free Two-Dimensional Barcode and Method for Printing and Reading the Samexe2x80x9d, (Ser. No. 09/088,189, filed Jun. 1, 1998) (xe2x80x9cthe ""189 Applicationxe2x80x9d), the contents of which are explicitly incorporated by reference herein, a clock-less two-dimensional barcode without a border (shown in FIG. 2 herein) is described, along with methods of printing and reading the same. Two alternate symbologies for the barcode are presented in the ""189 Application, a first symbology which requires that the four corner bits 21 to be black (when printed on a white background), and a second symbology in which no black corner bits 21 are required. As such, two alternate methods for reading the barcode of FIG. 2 are described in the ""189 Application, a first method which operates on the barcode which does not require corner bits, as described by the flowchart in FIG. 8A therein and the description related thereto, and a second method which operates on the barcode which is required to have corner bits, as described by the flowchart in FIG. 8B therein and the description related thereto. Although the two methods of reading the barcode described in the ""189 Application provide satisfactory results, it was found that when the barcode was printed on a page with a complex background, the results provided by the locate step 70 of FIGS. 8A and 8B of the ""189 Application, which is described therein in conjunction with FIGS. 9A and 9B, were less than optimal, particularly in the presence of single line noise conditions (i.e., an arbitrarily line across the barcode having a width less than or equal to the width of a bit block within the barcode, which can often occur in faxed documents and documents printed by poorly maintained printers). In addition, it was found that changes in the Hough Transform skew angle estimation step 71 of FIGS. 8A and 8B of the ""189 Application could be made to increase processing speed. Also, because of the increased processing speed of the Hough Transform skew estimation step of the present invention, the template matching skew angle estimation step 71 of FIG. 8B of the ""189 Application, which requires that the barcode include comer bits, decreasing the number of bits that could be stored within the barcode, and has a less than optimal processing speed, is no longer required.
It is therefore an object of the present invention to provide a method of decoding information digitally encoded in the form of a border-less clock free two-dimensional barcode printed on a printed medium which is able to operate in the presence of complex backgrounds.
It is an additional object of this invention to provide a method of decoding information digitally encoded in the form of a border-less clock free two-dimensional barcode printed on a printed medium which has an improved processing speed.
It is yet a further object of this invention to provide a method of decoding information digitally encoded in the form of a border-less clock free two-dimensional barcode printed on a printed medium which does not include corner bits.
It is another object of this invention to provide a method of decoding information digitally encoded in the form of a two-dimensional barcode printed on a printed medium which may or may not include a border.
Various other objects, advantages and features of the present invention will become readily apparent from the ensuing detailed description and the novel features will be particularly pointed out in the appended claims.
These and other objectives are realized by a method of decoding randomized information printed on a human readable medium in the form of a bitmap of rows and columns of data pixels representing encoded data bits. Each of the data pixels has either a first or second color. The bitmap has a predetermined size and is surrounded by an outer region of pixels of predetermined substantially uniform color. A border of contrasting color may be present within the outer region. The human readable medium is first scanned to digitize the bitmap and then formatted to a pixel based grayscale representation. The pixel based grayscale representation is converted to a pixel based binary representation by setting a threshold intensity level based on the grayscale representation and converting pixels greater than or equal to the threshold to a first level, e.g., xe2x80x9c0xe2x80x9d, and pixels less than the threshold to a second level, e.g., xe2x80x9c1xe2x80x9d. The row and column boundaries of the digitized bitmap are located by moving a window across the pixel based binary representation in stepwise fashion in a predetermined pattern. At each step the portion of the representation which is encompassed by the window is tested to determine whether the portion conforms to one or more characteristics of the bitmap, and the boundaries of the digitized bitmap are set as the boundaries of the window if the portion does conform to the one or more characteristics of the bitmap. The skew angle of the digitized bitmap is determined, and if necessary, the digitized bitmap is deskewed so that the skew angle is reduced to substantially zero. The digitized bitmap is thereafter cropped and the binary data is read out from the digitized bitmap, thereby producing a one-dimensional array of digital data. Finally, the one-dimensional array is derandomized and error-correction is applied to produce a substantially error-free digital representation of the encoded information.
In one embodiment, the window used in the locating step comprises a core region corresponding to the predetermined size of the bitmap and a quiet region corresponding to the outer region. The testing comprises separately testing portions of the representation encompassed by the core region and the quiet region to determine whether the portions conform to one or more characteristics of the bitmap and the outer region, respectively. Preferably, the pixel distribution of each region is tested to determine whether it falls within predetermined ranges to verify that the bitmap is present within the image, i.e., the bitmap within the core region will have an approximately even pixel distribution and the outer region will have a pixel distribution that has pixels of close to 100% of either xe2x80x9c0xe2x80x9d or xe2x80x9c1xe2x80x9d. If the portions of the representation encompassed by the core region and the quiet region conform to the one or more characteristics of the bitmap, the boundaries of a candidate region for the digitized bitmap are set to the boundaries of the core region. In addition, if the portions of the representation encompassed by the window are found to satisfy the previous testing, the portion encompassed by the core region may also be cropped to determine the outer boundaries of the candidate bitmap therein, and the outer boundaries compared to the predetermined dimensions of the bitmap, to further verify that a bitmap is present within the window. Further, when the core region and quiet region testing indicates that the portion of the region confirms to one or more characteristics of the bitmap, but the cropping test indicates that no bitmap is present, the window is advanced to completely avoid the object identified in the cropping test, to improve the speed of operation by eliminating further testing of that object.
In another embodiment of the present invention, the skew angle is determined by first locating all of the horizontal or vertical edges within the located candidate region, preferably using a finite-state recognizer. The coordinates of a horizontal or vertical line within the located candidate region representing the horizontal or vertical edges are then calculated using the Hough Transform. Finally, the skew angle is calculated as the angle between the coordinates of the horizontal or vertical line within the candidate region and a horizontal line representing a row of pixels within the representation or a vertical line representing a column of pixels within the candidate region. Optionally both the horizontal and vertical edges can be located, and the skew angle can be calculated using both the horizontal and vertical edges.
In yet another embodiment, the candidate region is divided into a plurality of horizontal and/or vertical regions. Preliminary skew angles are calculated for each of the plurality of horizontal and/or vertical regions, and the skew angle is selected by a voting scheme from the preliminary skew angles, e.g., the median value is selected.